Simultaneous Control and Human Feedback in the Training of a Robotic Agent with Actor-Critic Reinforcement Learning

This paper contributes a preliminary report on the advantages and disadvantages of incorporating simultaneous human control and feedback signals in the training of a reinforcement learning robotic agent. While robotic human-machine interfaces have become increasingly complex in both form and function, control remains challenging for users. This has resulted in an increasing gap between user control approaches and the number of robotic actu-ators which can be controlled. One way to address this gap is to shift autonomy to the robot. Semi-autonomous actions of the robotic agent can then be shaped and refined by human feedback, simplifying user control. Most prior work on human-agent shaping has incorporated training with feedback , or has included indirect control signals. By contrast, in this paper we explore how a human can provide concurrent feedback signals and real-time myoelectric control signals to train a robot's actor-critic reinforcement learning control system. Using both a physical and a simulated robotic system, we compare training performance on a simple movement task when reward is derived from the environment , from the human, and from the combination of the two. Our results indicate that benefit may be gained by including human generated feedback in learning algorithms for this complex human machine interactive domain.